Joint online distance learning to complement postgraduate pathology training in preparation for national board examinations.

AIMS: To meet the flexible learning needs of pathology residents preparing for national board examinations, a joint distance learning approach was developed using both asynchronous and synchronous activities with whole slide images, drawing on empirical educational research on online distance learning. METHODS: In a case study of an implementation of the designed joint distance learning approach with a geographically dispersed group of pathology residents in Finland, the participants' perceptions were measured with a 12-item questionnaire covering the value of the learning opportunity, the quality of the sociocognitive processes and their emotional engagement and social cohesion. Communication during the online session was also recorded and analysed to provide objectivity to the self-report data. RESULTS: The effectiveness of joint online learning for knowledge acquisition and preparation for national board examinations was highly rated. However, despite strong emotional engagement during synchronous activities, participants reported minimal interpersonal interaction, which was also reflected in the recordings of the online session. CONCLUSION: Using a technology integration framework and guided by the principles of self-determination theory, joint distance learning is emerging as a beneficial addition to postgraduate pathology programmes in preparation for national examinations. However, to realise the full potential of interpersonal interaction, participants should be prepared for an appropriate mindset.

Bringing AI to the clinic: blueprint for a vendor-neutral AI deployment infrastructure.

AI provides tremendous opportunities for improving patient care, but at present there is little evidence of real-world uptake. An important barrier is the lack of well-designed, vendor-neutral and future-proof infrastructures for deployment. Because current AI algorithms are very narrow in scope, it is expected that a typical hospital will deploy many algorithms concurrently. Managing stand-alone point solutions for all of these algorithms will be unmanageable. A solution to this problem is a dedicated platform for deployment of AI. Here we describe a blueprint for such a platform and the high-level design and implementation considerations of such a system that can be used clinically as well as for research and development. Close collaboration between radiologists, data scientists, software developers and experts in hospital IT as well as involvement of patients is crucial in order to successfully bring AI to the clinic.

Determinants of Difficulty and Discriminating Power of Image-based Test Items in Postgraduate Radiological Examinations.

RATIONALE AND OBJECTIVES: The psychometric characteristics of image-based test items in radiological written examinations are not well known. In this study, we explored difficulty and discriminating power of these test items in postgraduate radiological digital examinations. MATERIALS AND METHODS: We reviewed test items of seven Dutch Radiology Progress Tests (DRPTs) that were taken from October 2013 to April 2017. The DRPT is a semiannual formative examination, required for all Dutch radiology residents. We assessed several stimulus and response characteristics of test items. The response format of test items included true or false, single right multiple choice with 2, 3, 4, or ≥5 answer options, pick-N multiple-choice, drag-and-drop, and long-list-menu formats. We calculated item P values and item-rest-correlation (Rir) values to assess difficulty and discriminating power. We performed linear regression analysis in image-based test items to investigate whether P and Rir values were significantly related to stimulus and response characteristics. Also, we compared psychometric indices between image-based test items and text-alone items. RESULTS: P and Rir values of image-based items (n = 369) were significantly related to the type of response format (P < .001), and not to which of the seven DRPTs the item was obtained from, radiological subspecialty domain, nonvolumetric or volumetric character of images, or context-rich or context-free character of the stimulus. When accounted for type of response format, difficulty and discriminating power of image-based items did not differ significantly from text-alone items (n = 881). Test items with a relatively large number of answer options were generally more difficult, and discriminated better among high- and low-performing candidates. CONCLUSION: In postgraduate radiological written examinations, difficulty and discriminating power of image-based test items are related to the type of response format and are comparable to those of text-alone items. We recommend a response format with a relatively large number of answer options to optimize psychometric indices of radiological image-based test items.

Increasing Authenticity of Simulation-Based Assessment in Diagnostic Radiology.

INTRODUCTION: Clinical reasoning in diagnostic imaging professions is a complex skill that requires processing of visual information and image manipulation skills. We developed a digital simulation-based test method to increase authenticity of image interpretation skill assessment. METHODS: A digital application, allowing volumetric image viewing and manipulation, was used for three test administrations of the national Dutch Radiology Progress Test for residents. This study describes the development and implementation process in three phases. To assess authenticity of the digital tests, perceived image quality and correspondence to clinical practice were evaluated and compared with previous paper-based tests (PTs). Quantitative and qualitative evaluation results were used to improve subsequent tests. RESULTS: Authenticity of the first digital test was not rated higher than the PTs. Test characteristics and environmental conditions, such as image manipulation options and ambient lighting, were optimized based on participants' comments. After adjustments in the third digital test, participants favored the image quality and clinical correspondence of the digital image questions over paper-based image questions. CONCLUSIONS: Digital simulations can increase authenticity of diagnostic radiology assessments compared with paper-based testing. However, authenticity does not necessarily increase with higher fidelity. It can be challenging to simulate the image interpretation task of clinical practice in a large-scale assessment setting, because of technological limitations. Optimizing image manipulation options, the level of ambient light, time limits, and question types can help improve authenticity of simulation-based radiology assessments.

Identifying error types in visual diagnostic skill assessment.

BACKGROUND: Misinterpretation of medical images is an important source of diagnostic error. Errors can occur in different phases of the diagnostic process. Insight in the error types made by learners is crucial for training and giving effective feedback. Most diagnostic skill tests however penalize diagnostic mistakes without an eye for the diagnostic process and the type of error. A radiology test with stepwise reasoning questions was used to distinguish error types in the visual diagnostic process. We evaluated the additional value of a stepwise question-format, in comparison with only diagnostic questions in radiology tests. METHODS: Medical students in a radiology elective (n=109) took a radiology test including 11-13 cases in stepwise question-format: marking an abnormality, describing the abnormality and giving a diagnosis. Errors were coded by two independent researchers as perception, analysis, diagnosis, or undefined. Erroneous cases were further evaluated for the presence of latent errors or partial knowledge. Inter-rater reliabilities and percentages of cases with latent errors and partial knowledge were calculated. RESULTS: The stepwise question-format procedure applied to 1351 cases completed by 109 medical students revealed 828 errors. Mean inter-rater reliability of error type coding was Cohen's κ=0.79. Six hundred and fifty errors (79%) could be coded as perception, analysis or diagnosis errors. The stepwise question-format revealed latent errors in 9% and partial knowledge in 18% of cases. CONCLUSIONS: A stepwise question-format can reliably distinguish error types in the visual diagnostic process, and reveals latent errors and partial knowledge.

Support for external validity of radiological anatomy tests using volumetric images.

RATIONALE AND OBJECTIVES: Radiology practice has become increasingly based on volumetric images (VIs), but tests in medical education still mainly involve two-dimensional (2D) images. We created a novel, digital, VI test and hypothesized that scores on this test would better reflect radiological anatomy skills than scores on a traditional 2D image test. To evaluate external validity we correlated VI and 2D image test scores with anatomy cadaver-based test scores. MATERIALS AND METHODS: In 2012, 246 medical students completed one of two comparable versions (A and B) of a digital radiology test, each containing 20 2D image and 20 VI questions. Thirty-three of these participants also took a human cadaver anatomy test. Mean scores and reliabilities of the 2D image and VI subtests were compared and correlated with human cadaver anatomy test scores. Participants received a questionnaire about perceived representativeness and difficulty of the radiology test. RESULTS: Human cadaver test scores were not correlated with 2D image scores, but significantly correlated with VI scores (r = 0.44, P < .05). Cronbach's α reliability was 0.49 (A) and 0.65 (B) for the 2D image subtests and 0.65 (A) and 0.71 (B) for VI subtests. Mean VI scores (74.4%, standard deviation 2.9) were significantly lower than 2D image scores (83.8%, standard deviation 2.4) in version A (P < .001). VI questions were considered more representative of clinical practice and education than 2D image questions and less difficult (both P < .001). CONCLUSIONS: VI tests show higher reliability, a significant correlation with human cadaver test scores, and are considered more representative for clinical practice than tests with 2D images.

Volumetric CT-images improve testing of radiological image interpretation skills.

RATIONALE AND OBJECTIVES: Current radiology practice increasingly involves interpretation of volumetric data sets. In contrast, most radiology tests still contain only 2D images. We introduced a new testing tool that allows for stack viewing of volumetric images in our undergraduate radiology program. We hypothesized that tests with volumetric CT-images enhance test quality, in comparison with traditional completely 2D image-based tests, because they might better reflect required skills for clinical practice. MATERIALS AND METHODS: Two groups of medical students (n=139; n=143), trained with 2D and volumetric CT-images, took a digital radiology test in two versions (A and B), each containing both 2D and volumetric CT-image questions. In a questionnaire, they were asked to comment on the representativeness for clinical practice, difficulty and user-friendliness of the test questions and testing program. Students' test scores and reliabilities, measured with Cronbach's alpha, of 2D and volumetric CT-image tests were compared. RESULTS: Estimated reliabilities (Cronbach's alphas) were higher for volumetric CT-image scores (version A: .51 and version B: .54), than for 2D CT-image scores (version A: .24 and version B: .37). Participants found volumetric CT-image tests more representative of clinical practice, and considered them to be less difficult than volumetric CT-image questions. However, in one version (A), volumetric CT-image scores (M 80.9, SD 14.8) were significantly lower than 2D CT-image scores (M 88.4, SD 10.4) (p<.001). The volumetric CT-image testing program was considered user-friendly. CONCLUSION: This study shows that volumetric image questions can be successfully integrated in students' radiology testing. Results suggests that the inclusion of volumetric CT-images might improve the quality of radiology tests by positively impacting perceived representativeness for clinical practice and increasing reliability of the test.

Evaluation of automatic neonatal brain segmentation algorithms: the NeoBrainS12 challenge.

A number of algorithms for brain segmentation in preterm born infants have been published, but a reliable comparison of their performance is lacking. The NeoBrainS12 study (http://neobrains12.isi.uu.nl), providing three different image sets of preterm born infants, was set up to provide such a comparison. These sets are (i) axial scans acquired at 40 weeks corrected age, (ii) coronal scans acquired at 30 weeks corrected age and (iii) coronal scans acquired at 40 weeks corrected age. Each of these three sets consists of three T1- and T2-weighted MR images of the brain acquired with a 3T MRI scanner. The task was to segment cortical grey matter, non-myelinated and myelinated white matter, brainstem, basal ganglia and thalami, cerebellum, and cerebrospinal fluid in the ventricles and in the extracerebral space separately. Any team could upload the results and all segmentations were evaluated in the same way. This paper presents the results of eight participating teams. The results demonstrate that the participating methods were able to segment all tissue classes well, except myelinated white matter.

Cardiac valve calcifications on low-dose unenhanced ungated chest computed tomography: inter-observer and inter-examination reliability, agreement and variability.

OBJECTIVES: To determine inter-observer and inter-examination variability for aortic valve calcification (AVC) and mitral valve and annulus calcification (MC) in low-dose unenhanced ungated lung cancer screening chest computed tomography (CT). METHODS: We included 578 lung cancer screening trial participants who were examined by CT twice within 3 months to follow indeterminate pulmonary nodules. On these CTs, AVC and MC were measured in cubic millimetres. One hundred CTs were examined by five observers to determine the inter-observer variability. Reliability was assessed by kappa statistics (κ) and intra-class correlation coefficients (ICCs). Variability was expressed as the mean difference ± standard deviation (SD). RESULTS: Inter-examination reliability was excellent for AVC (κ = 0.94, ICC = 0.96) and MC (κ = 0.95, ICC = 0.90). Inter-examination variability was 12.7 ± 118.2 mm(3) for AVC and 31.5 ± 219.2 mm(3) for MC. Inter-observer reliability ranged from κ = 0.68 to κ = 0.92 for AVC and from κ = 0.20 to κ = 0.66 for MC. Inter-observer ICC was 0.94 for AVC and ranged from 0.56 to 0.97 for MC. Inter-observer variability ranged from -30.5 ± 252.0 mm(3) to 84.0 ± 240.5 mm(3) for AVC and from -95.2 ± 210.0 mm(3) to 303.7 ± 501.6 mm(3) for MC. CONCLUSIONS: AVC can be quantified with excellent reliability on ungated unenhanced low-dose chest CT, but manual detection of MC can be subject to substantial inter-observer variability. Lung cancer screening CT may be used for detection and quantification of cardiac valve calcifications. KEY POINTS: • Low-dose unenhanced ungated chest computed tomography can detect cardiac valve calcifications. • However, calcified cardiac valves are not reported by most radiologists. • Inter-observer and inter-examination variability of aortic valve calcifications is sufficient for longitudinal studies. • Volumetric measurement variability of mitral valve and annulus calcifications is substantial.

Automated coronary artery calcification scoring in non-gated chest CT: agreement and reliability.

OBJECTIVE: To determine the agreement and reliability of fully automated coronary artery calcium (CAC) scoring in a lung cancer screening population. MATERIALS AND METHODS: 1793 low-dose chest CT scans were analyzed (non-contrast-enhanced, non-gated). To establish the reference standard for CAC, first automated calcium scoring was performed using a preliminary version of a method employing coronary calcium atlas and machine learning approach. Thereafter, each scan was inspected by one of four trained raters. When needed, the raters corrected initially automaticity-identified results. In addition, an independent observer subsequently inspected manually corrected results and discarded scans with gross segmentation errors. Subsequently, fully automatic coronary calcium scoring was performed. Agatston score, CAC volume and number of calcifications were computed. Agreement was determined by calculating proportion of agreement and examining Bland-Altman plots. Reliability was determined by calculating linearly weighted kappa (κ) for Agatston strata and intraclass correlation coefficient (ICC) for continuous values. RESULTS: 44 (2.5%) scans were excluded due to metal artifacts or gross segmentation errors. In the remaining 1749 scans, median Agatston score was 39.6 (P25-P75∶0-345.9), median volume score was 60.4 mm3 (P25-P75∶0-361.4) and median number of calcifications was 2 (P25-P75∶0-4) for the automated scores. The κ demonstrated very good reliability (0.85) for Agatston risk categories between the automated and reference scores. The Bland-Altman plots showed underestimation of calcium score values by automated quantification. Median difference was 2.5 (p25-p75∶0.0-53.2) for Agatston score, 7.6 (p25-p75∶0.0-94.4) for CAC volume and 1 (p25-p75∶0-5) for number of calcifications. The ICC was very good for Agatston score (0.90), very good for calcium volume (0.88) and good for number of calcifications (0.64). DISCUSSION: Fully automated coronary calcium scoring in a lung cancer screening setting is feasible with acceptable reliability and agreement despite an underestimation of the amount of calcium when compared to reference scores.

Automatic segmentation of eight tissue classes in neonatal brain MRI.

PURPOSE: Volumetric measurements of neonatal brain tissues may be used as a biomarker for later neurodevelopmental outcome. We propose an automatic method for probabilistic brain segmentation in neonatal MRIs. MATERIALS AND METHODS: In an IRB-approved study axial T1- and T2-weighted MR images were acquired at term-equivalent age for a preterm cohort of 108 neonates. A method for automatic probabilistic segmentation of the images into eight cerebral tissue classes was developed: cortical and central grey matter, unmyelinated and myelinated white matter, cerebrospinal fluid in the ventricles and in the extra cerebral space, brainstem and cerebellum. Segmentation is based on supervised pixel classification using intensity values and spatial positions of the image voxels. The method was trained and evaluated using leave-one-out experiments on seven images, for which an expert had set a reference standard manually. Subsequently, the method was applied to the remaining 101 scans, and the resulting segmentations were evaluated visually by three experts. Finally, volumes of the eight segmented tissue classes were determined for each patient. RESULTS: The Dice similarity coefficients of the segmented tissue classes, except myelinated white matter, ranged from 0.75 to 0.92. Myelinated white matter was difficult to segment and the achieved Dice coefficient was 0.47. Visual analysis of the results demonstrated accurate segmentations of the eight tissue classes. The probabilistic segmentation method produced volumes that compared favorably with the reference standard. CONCLUSION: The proposed method provides accurate segmentation of neonatal brain MR images into all given tissue classes, except myelinated white matter. This is the one of the first methods that distinguishes cerebrospinal fluid in the ventricles from cerebrospinal fluid in the extracerebral space. This method might be helpful in predicting neurodevelopmental outcome and useful for evaluating neuroprotective clinical trials in neonates.

Intra and interobserver reliability and agreement of semiquantitative vertebral fracture assessment on chest computed tomography.

OBJECTIVES: To evaluate the reliability of semiquantitative Vertebral Fracture Assessment (VFA) on chest Computed Tomography (CT). METHODS: Four observers performed VFA twice upon sagittal reconstructions of 50 routine clinical chest CTs. Intra- and interobserver agreement (absolute agreement or 95% Limits of Agreement) and reliability (Cohen's kappa or intraclass correlation coefficient(ICC)) were calculated for the visual VFA measures (fracture present, worst fracture grade, cumulative fracture grade on patient level) and for percentage height loss of each fractured vertebra compared to the adjacent vertebrae. RESULTS: Observers classified 24-38% patients as having at least one vertebral fracture, giving rise to kappa's of 0.73-0.84 (intraobserver) and 0.56-0.81 (interobserver). For worst fracture grade we found good intraobserver (76-88%) and interobserver (74-88%) agreement, and excellent reliability with square-weighted kappa's of 0.84-0.90 (intraobserver) and 0.84-0.94 (interobserver). For cumulative fracture grade the 95% Limits of Agreement were maximally ±1,99 (intraobserver) and ±2,69 (interobserver) and the reliability (ICC) varied from 0.84-0.94 (intraobserver) and 0.74-0.94 (interobserver). For percentage height-loss on a vertebral level the 95% Limits of Agreement were maximally ±11,75% (intraobserver) and ±12,53% (interobserver). The ICC was 0.59-0.90 (intraobserver) and 0.53-0-82 (interobserver). Further investigation is needed to evaluate the prognostic value of this approach. CONCLUSION: In conclusion, these results demonstrate acceptable reproducibility of VFA on CT.

Variation in quantitative CT air trapping in heavy smokers on repeat CT examinations.

OBJECTIVES: To determine the variation in quantitative computed tomography (CT) measures of air trapping in low-dose chest CTs of heavy smokers. METHODS: We analysed 45 subjects from a lung cancer screening trial, examined by CT twice within 3 months. Inspiratory and expiratory low-dose CT was obtained using breath hold instructions. CT air trapping was defined as the percentage of voxels in expiratory CT with an attenuation below -856 HU (EXP(-856)) and the expiratory to inspiratory ratio of mean lung density (E/I-ratio(MLD)). Variation was determined using limits of agreement, defined as 1.96 times the standard deviation of the mean difference. The effect of both lung volume correction and breath hold reproducibility was determined. RESULTS: The limits of agreement for uncorrected CT air trapping measurements were -15.0 to 11.7 % (EXP(-856)) and -9.8 to 8.0 % (E/I-ratio(MLD)). Good breath hold reproducibility significantly narrowed the limits for EXP(-856) (-10.7 to 7.5 %, P = 0.002), but not for E/I-ratio(MLD) (-9.2 to 7.9 %, P = 0.75). Statistical lung volume correction did not improve the limits for EXP(-856) (-12.5 to 8.8 %, P = 0.12) and E/I-ratio(MLD) (-7.5 to 5.8 %, P = 0.17). CONCLUSIONS: Quantitative air trapping measures on low-dose CT of heavy smokers show considerable variation on repeat CT examinations, regardless of lung volume correction or reproducible breath holds. KEY POINTS: Computed tomography quantitatively measures small airways disease in heavy smokers. Measurements of air trapping vary considerably on repeat CT examinations. Variation remains substantial even with reproducible breath holds and lung volume correction.

The effect of iterative reconstruction on computed tomography assessment of emphysema, air trapping and airway dimensions.

OBJECTIVES: To determine the influence of iterative reconstruction (IR) on quantitative computed tomography (CT) measurements of emphysema, air trapping, and airway wall and lumen dimensions, compared to filtered back-projection (FBP). METHODS: Inspiratory and expiratory chest CTs of 75 patients (37 male, 38 female; mean age 64.0 ± 5.7 years) were reconstructed using FBP and IR. CT emphysema, CT air trapping and airway dimensions of a segmental bronchus were quantified using several commonly used quantification methods. The two algorithms were compared using the concordance correlation coefficient (p (c)) and Wilcoxon signed rank test. RESULTS: Only the E/I-ratio(MLD) as a measure of CT air trapping and airway dimensions showed no significant differences between the algorithms, whereas all CT emphysema and the other CT air trapping measures were significantly different at IR when compared to FBP (P < 0.001). CONCLUSION: The evaluated IR algorithm significantly influences quantitative CT measures in the assessment of emphysema and air trapping. However, the E/I-ratio(MLD) as a measure of CT air trapping, as well as the airway measurements, is unaffected by this reconstruction method. Quantitative CT of the lungs should be performed with careful attention to the CT protocol, especially when iterative reconstruction is introduced. KEY POINTS : • New techniques in CT allow numerous quantitative measurements of lung function. • Iterative reconstruction influences quantitative CT measurements of emphysema and air trapping. • Expiratory-to-inspiratory ratio of mean lung density and airway measurements are unaffected by iterative reconstruction. • Quantitative lung-CT should be performed with careful attention to the CT protocol.

CT-quantified emphysema in male heavy smokers: association with lung function decline.

BACKGROUND: Emphysema and small airway disease both contribute to chronic obstructive pulmonary disease (COPD), a disease characterised by accelerated decline in lung function. The association between the extent of emphysema in male current and former smokers and lung function decline was investigated. METHODS: Current and former heavy smokers participating in a lung cancer screening trial were recruited to the study and all underwent CT. Spirometry was performed at baseline and at 3-year follow-up. The 15th percentile (Perc15) was used to assess the severity of emphysema. RESULTS: 2085 men of mean age 59.8 years participated in the study. Mean (SD) baseline Perc15 was -934.9 (19.5) HU. A lower Perc15 value correlated with a lower forced expiratory volume in 1 s (FEV(1)) at baseline (r=0.12, p<0.001). Linear mixed model analysis showed that a lower Perc15 was significantly related to a greater decline in FEV(1) after follow-up (p<0.001). Participants without baseline airway obstruction who developed it after follow-up had significantly lower mean (SD) Perc15 values at baseline than those who did not develop obstruction (-934.2 (17.1) HU vs -930.2 (19.7) HU, p<0.001). CONCLUSION: Greater baseline severity of CT-detected emphysema is related to lower baseline lung function and greater rates of lung function decline, even in those without airway obstruction. CT-detected emphysema aids in identifying non-obstructed male smokers who will develop airflow obstruction.

Fusion of local and global detection systems to detect tuberculosis in chest radiographs.

Automatic detection of tuberculosis (TB) on chest radiographs is a difficult problem because of the diverse presentation of the disease. A combination of detection systems for abnormalities and normal anatomy is used to improve detection performance. A textural abnormality detection system operating at the pixel level is combined with a clavicle detection system to suppress false positive responses. The output of a shape abnormality detection system operating at the image level is combined in a next step to further improve performance by reducing false negatives. Strategies for combining systems based on serial and parallel configurations were evaluated using the minimum, maximum, product, and mean probability combination rules. The performance of TB detection increased, as measured using the area under the ROC curve, from 0.67 for the textural abnormality detection system alone to 0.86 when the three systems were combined. The best result was achieved using the sum and product rule in a parallel combination of outputs.